ABSTRACT
Wearing a facial mask has become a must in our daily life due to the global COVID-19 pandemic. However, the performance of conventional face recognition systems severely degrades for faces occluded by masks. How to combat the effect of occlusion on face recognition is an important issue. However, the performance of existing methods developed for masked face recognition unpleasantly degrades when dealing with unmasked faces. To address this issue for real-world applications, where the gallery image or the probe image may be a masked or unmasked face, we propose the concept of balanced facial feature matching and, based on it, design a robust masked face recognition system. The matching is balanced because it is performed on features extracted from corresponding facial regions. The system consists of a classification network and two feature extractors. The classification network classifies an input face image into a masked face or an unmasked face. One feature extractor extracts the feature of a full face, and the other uses a guided perceptual loss to focus the feature extraction on the non-occluded part of the face. The system is tested on both synthetic and real data. The face verification accuracy is improved by 2.4% for the synthetically masked LFW dataset, 1.9% for the MFR2 dataset, and 5.4% for the RMFD dataset. The results further show that the system improves masked face recognition while preserving the performance of unmasked face recognition. © 2022 IEEE.